update: Sail to 0.6.6#982
Conversation
| SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime, SearchPhrase LIMIT 10; | ||
| SELECT CounterID, AVG(length(URL)) AS l, COUNT(*) AS c FROM hits WHERE URL <> '' GROUP BY CounterID HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; | ||
| SELECT REGEXP_REPLACE(Referer, '^https?://(?:www\.)?([^/]+)/.*$', '\1') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; | ||
| SELECT REGEXP_REPLACE(Referer, '^https?://(?:www\.)?([^/]+)/.*$', '$1') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; |
There was a problem hiding this comment.
Query was previously returning incorrect results. Should be $1.
| os.environ["SAIL_PARQUET__BINARY_AS_STRING"] = "true" | ||
| os.environ["SAIL_PARQUET__REORDER_FILTERS"] = "true" | ||
| os.environ["SAIL_OPTIMIZER__ENABLE_JOIN_REORDER"] = "true" |
There was a problem hiding this comment.
This does nothing since ClickBench has no joins
|
|
||
| os.environ["SAIL_PARQUET__BINARY_AS_STRING"] = "true" | ||
| os.environ["SAIL_PARQUET__REORDER_FILTERS"] = "true" | ||
| os.environ["SAIL_OPTIMIZER__ENABLE_JOIN_REORDER"] = "true" |
There was a problem hiding this comment.
This does nothing since ClickBench has no joins
| SELECT SearchPhrase FROM hits WHERE SearchPhrase <> '' ORDER BY EventTime, SearchPhrase LIMIT 10; | ||
| SELECT CounterID, AVG(length(URL)) AS l, COUNT(*) AS c FROM hits WHERE URL <> '' GROUP BY CounterID HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; | ||
| SELECT REGEXP_REPLACE(Referer, '^https?://(?:www\.)?([^/]+)/.*$', '\1') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; | ||
| SELECT REGEXP_REPLACE(Referer, '^https?://(?:www\.)?([^/]+)/.*$', '$1') AS k, AVG(length(Referer)) AS l, COUNT(*) AS c, MIN(Referer) FROM hits WHERE Referer <> '' GROUP BY k HAVING COUNT(*) > 100000 ORDER BY l DESC LIMIT 25; |
There was a problem hiding this comment.
Query was previously returning incorrect results. Should be $1.
| spark = SparkSession.builder.remote(f"sc://localhost:{port}").getOrCreate() | ||
| df = spark.read.parquet("partitioned") | ||
| df = df.withColumn("EventTime", F.col("EventTime").cast("timestamp")) |
There was a problem hiding this comment.
I added this in #967, turns out it is unnecessary.
| spark = SparkSession.builder.remote(f"sc://localhost:{port}").getOrCreate() | ||
|
|
||
| df = spark.read.parquet("hits.parquet") | ||
| df = df.withColumn("EventTime", F.col("EventTime").cast("timestamp")) |
There was a problem hiding this comment.
I added this in #967, turns out it is unnecessary.
|
ClickBench currently measures Sail 0.6.5. This PR upgrades to 0.6.6, released one week after 0.6.5. Did anything related to performance change in this seemingly minor release? Would it be possible to postpone the next ClickBench submission until the next major release? Asking because creating measurements (like here just a couple of days ago) creates some manual work on our end. |
|
@rschu1ze Yeah we can wait until the big Sail release. I'll leave this PR open for now and ping you when it's updated |
GitHub Actions
Inactive
|
Results for Logs:
|
|
Results for Logs:
|
…large, c6a.metal, c6a.xlarge, c7a.metal-48xl, c8g.4xlarge, c8g.metal-48xl)
GitHub Actions
Waiting
|
The result files from the comment above are now committed to this PR directly (d6de6f2) — no need to save them manually. This was a test of new automation (#1011) that lets the results collector push to fork PRs that allow maintainer edits. 🤖 Generated with Claude Code |
4 participants
No description provided.